Coded video (e.g., MPEG-2 (for Moving Picture Experts Group) and H.264/AVC (for Advanced Video Coding)) is used in Digital Television (DTV) networks for Broadcast Television and On-Demand video. The digital format enables the coded video to be displayed on a television (e.g., cathode ray tube (CRT), liquid crystal display (LCD), and Plasma) once decoded, as well as downloaded and saved to a disk drive for display at a later time (e.g., on a computer, Digital Recorder, etc). When a program is coded, its video and audio are separated into Elementary Streams (ES). Each ES is segmented into packets that form a Packetized Elementary Stream (PES). In order to transport and display the video in real-time, the PES is further segmented and encapsulated by an MPEG-2 Transport Stream (TS). The MPEG-2 TS contains timing information (i.e., Program Clock Reference (PCR), Presentation Time-Stamp (PTS), and Decode Time-Stamp (DTS)) that is used by a video decoder to synchronize with an encoder. Timing information is also used to transport video across Quadrature Amplitude Modulation (QAM) Networks (such as in Cable TV), over Data Over Cable Service Interface Specification (DOCSIS) (e.g., in an Internet Protocol Television (IPTV) deployment), and over Layer-2 Packet Networks that often form the infrastructure connecting video elements together.
The MPEG-2 Transport Stream (TS) consists of fixed size 188-byte packets, each with a 4-byte header that describes the 184-byte TS payload. The TS Header includes a 13-bit Program ID (PID) that is used to uniquely identify each TS packet and payload. The MPEG-2 TS is able to carry many different programs simultaneously. When multiple programs are present in the TS, the TS is called a Multiple Program Transport Stream (MPTS). A Single Program Transport Stream (SPTS) carries only a single program. In addition to carrying the video and audio PES, the TS also carries program specific information that is contained in a Program Association Table (PAT) and a Program Mapping table (PMT). These associate the PID with the actual program.
When an MPEG-2 TS is carried across a Packet Network (e.g., Layer-2 or Layer-3), the MPEG-2 TS is sometimes encapsulated with the Real-time Transport Protocol (RTP). RTP provides a means for assigning a Timestamp that can be used for timing the transmission of Packets through a Network, such as from a Video Server to a Video or Edge QAM in a Cable TV network. The RTP Timestamp has a fixed relationship to the PCR/PTS/DTS in the MPEG-2 TS, which is used to avoid overrunning or under-running a buffer at the receiver (e.g., Set Top Box (STB)). Anytime the relationship between the video and RTP Timestamp in the RTP packet is changed (e.g., repacketizing) the RTP Timestamp needs to be recomputed based on the PCR in the MPEG-2 TS.
Transport Streams (TS) may be of a Variable Bit Rate (VBR) or a Constant Bit Rate (CBR). In a VBR stream the rate can vary over time, generally up to a “peak” or “capped” rate. Although the rate of a VBR stream may vary, the video and audio must maintain a fixed relationship to its TS timing to correctly decode and display the video and audio (e.g., without underflow or overflow of its decoder buffer). A CBR stream can be transmitted in a similar manner to maintain its timing relationships, but at a constant rate that is maintained by inserting TS Null packets (PID 0x1FFF) into the stream when no video is present. CBR is used, for example, in Cable TV deployments where QAM channels are used to carry the video and audio. VBR is used, for example, in IPTV deployments, where the MPEG-2 TS is carried across IP Networks (including over DOCSIS in a Cable TV deployment, which also uses one or more QAMs).
The timing relationship between the coded audio and video, and the PCR/PTS/DTS in the MPEG-2 TS, is similar whether the video is transported as VBR or CBR. From a decoder perspective, the video and audio should be available in its decoder buffer at the correct time in order to be correctly displayed. If, for example, the video or audio arrives too early then the decoder buffer may overfill and data will be lost. If the video or audio arrives too late, the buffer may underflow and the video may miss its decoder frame time (e.g., DTS/PTS at 30 fps) and the decoder will need to repeat a previous frame on the display.
When an MPTS that is coded as CBR is received (e.g., from a video server) and the program elements need to be separated into multiple individual SPTS, all TS Null packets in the MPTS are removed. Since the TS Null packets have a common PID (0x1FFF), it is not possible to associate them with any particular SPTS. So they are discarded before storing the SPTS to a buffer. If the MPTS is carried using RTP, the Timestamp can be used when creating RTP Packets for each SPTS, if the timing relationship is maintained. If the inter-packet timing relationship is changed, then a new RTP Timestamp is computed using the PCR in the MPEG-2 TS.
FIG. 1 is an exemplary diagram 100 of the prior art illustrating MPTS packets of MPTS 101 carried using RTP (e.g., received from a network). Diagram 100 includes MPTS Pkt #0 102A, MPTS Pkt #1 102B, MPTS Pkt #2 102C, MPTS Pkt #3 102D, MPTS Pkt #4 102E, MPTS Pkt #5 102F, and MPTS Pkt #6 102G (collectively MPTS Pkts 102). Each of the MPTS Pkts 102 includes an RTP header (e.g., RTP header 104A through RTP header 104G, collectively RTP headers 104). MPTS PKT #0 102A includes PCR timestamps 106A, 106B, 106C and 106D. MPTS PKT #5 102F includes PCR timestamps 106E, 106F, 106G, and 106H.
MPTS PKT #0 102A includes SPTS A, #0 (i.e., SPTS for stream A, packet number 0), SPTS B, #0, SPTS C, #0, SPTS D, #0, SPTS D, #1, SPTS D, #2, and SPTS D, #3. MPTS PKT #1 102B includes SPTS B, #1, SPTS B, #2, SPTS B, #3, SPTS A, #1, SPTS A, #2, and SPTS A, #3. MPTS PKT #2 102C includes SPTS C, #1, SPTS B, #4, SPTS B, #5, SPTS D, #4, SPTS D, #5, SPTS D, #6, and SPTS D, #7. MPTS PKT #3 102D includes SPTS C, #2, SPTS C, #3, SPTS C, #4, SPTS C, #5, SPTS C, #6, SPTS A, #4, and SPTS A, #5. MPTS PKT #4 102E includes SPTS C, #7, SPTS C, #8, SPTS B, #6, SPTS B, #7, SPTS B, #8, SPTS A, #6, and SPTS A, #7. MPTS PKT #5 102F includes SPTS A, #8, SPTS B, #9, SPTS C, #9, SPTS D, #8, SPTS A, #9, SPTS A, #10, and SPTS B, #10. MPTS PKT #6 includes SPTS C, #10, SPTS B, #11, SPTS D, #9, and SPTS A, #11.
MPTS 101 includes packets multiplexed from four SPTSs, SPTS A 108A, SPTS 108B, SPTS 108C, and SPTS 108D, collectively SPTSs 108. SPTS A 108A includes RTP headers 110A, 110B, and 110C. STPS A 108A includes PCR timestamp 106A and PCR timestamp 106E. SPTS A 108A includes packets SPTS A, #1, SPTS A, #2, SPTS A, #3, SPTS A, #4, SPTS A, #5, SPTS A, #6, SPTS A, #7, SPTS A, #8, SPTS A, #9, SPTS A, #10, and SPTS A, #11. SPTS B 108B includes RTP headers 112A, 112B, and 112C. STPS B 108B includes PCR timestamp 106B and PCR timestamp 106F. SPTS B 108B includes packets SPTS B, #1, SPTS B, #2, SPTS B, #3, SPTS B, #4, SPTS B, #5, SPTS B, #6, SPTS B, #7, SPTS B, #8, SPTS B, #9, SPTS B, #10, and SPTS B, #11.
SPTS C 108C includes RTP headers 114A, 114B, and 114C. STPS C 108C includes PCR timestamp 106C and PCR timestamp 106G. SPTS C 108C includes packets SPTS C, #1, SPTS C, #2, SPTS C, #3, SPTS C, #4, SPTS C, #5, SPTS C, #6, SPTS C, #7, SPTS C, #8, SPTS C, #9, and SPTS C, #10. SPTS D 108D includes RTP headers 116A, 116B, and 116C. STPS D 108D includes PCR timestamp 106D and PCR timestamp 106H. SPTS D 108D includes packets SPTS D, #1, SPTS D, #2, SPTS D, #3, SPTS D, #4, SPTS D, #5, SPTS D, #6, SPTS D, #7, SPTS D, #8, and SPTS D, #9.
SPTSs 108 can be multiplexed into MPTS 101, and/or demultiplexed from MPTS 101 to SPTSs 108. MPTS 101 can be transported, for example, as CBR or VBR. For example, for CBR, TS Null packets (not shown) are inserted during multiplexing to maintain the MPTS rate, and the TS Null packets are discarded as each SPTS is de-multiplexed. The RTP Timestamp is recomputed as needed based on the PCR in each SPTS.
Like the MPTS 101, an SPTS (e.g., received from a network using RTP) can be transported as CBR or VBR. Unlike MPTS 101, an SPTS includes data for a single program. The RTP Timestamps of the SPTS can be computed based on the PCR in the MPEG-2 TS. Since the SPTS contains only a single video PES, the TS Null packets can be maintained to preserve the transport bit rate (i.e., the CBR). When the RTP Packets for the SPTS are received and stored, the MPEG-2 TS Null packets can be stored with the TS packets for the audio and video PES. The RTP Timestamp may also be preserved to avoid regeneration when the video is transmitted.
FIG. 2 is an exemplary diagram 120 of the prior art illustrating RTP packets for a SPTS, the RTP packets including one or more TS Null packets. Diagram 120 includes RTP packets 122A, 122B, and 122C, collectively RTP packets 122. Each of the RTP packets 122 includes an RTP header 124A, 124B, and 124C, respectively. RTP packet 122A includes TS packets TS 0, TS 1, TS 2, and TS 3. RTP packet 122A includes TS Null packets 126A, 126B, and 126C, collectively TS Null packets 126. RTP packet 122B includes TS 4, TS 5, TS 6, and TS 7. RTP packet 122B includes TS Null packets 128A, 128B, and 128C, collectively TS Null packets 128. RTP packet 122C includes TS 8, TS 9, TS 10, TS 11, TS 12, and TS 13. RTP packet 122C includes TS Null packet 130. The SPTS RTP packets 122 are received from the network and stored with the TS Null packets included.
Video that is coded and transported as CBR is generally done so for the convenience of the transmission system (e.g., QAM), as well as the receiver (e.g., STB). Since there are variations in image complexity of coded video, the coded information rate can vary over time. As such, a CBR stream may contain MPEG-2 TS Null packets (e.g., TS Null packets 126, 128, and 130) to fill in the SPTS when there is not enough video to maintain the CBR. Depending on the ratio of average to peak rate (i.e., the CBR) in the video stream, Null packets may account for 10% to 20% of the SPTS packets. For example, the SPTS of diagram 120 includes fourteen TS packets (TS 0 through TS 13) and seven TS Null packets (TS Null packets 126, 128, and 130), the TS Null packets accounting for 33.3% of the SPTS packets. If the Null packets are captured and stored, along with the PES, then that amount of storage capacity and additional bandwidth that is needed would be that much higher than necessary. This is especially true when the video is captured as CBR (e.g., captured and stored with both TS packets TS 0 through TS 13 and TS Null packets 126, 128, and 130) and then transmitted as VBR, and the TS Null packets are discarded first before delivery. Storing CBR and VBR versions of streams separately nearly doubles the storage requirement.
When video is captured and the MPEG-2 TS Nulls are removed for efficient storage, the MPEG-2 TS Nulls need to be recreated for CBR delivery. One solution requires packing the MPEG-TS packets into network packets, which is an expensive operation that uses the PCR in the MPEG-TS. This solution requires knowing the bit rate (i.e., CBR) of the stream and then having the ability to synthesize the TS Null packets at delivery for each stream. This costly operation directly impacts a systems ability to scale to many delivery streams, and the product cost associated with doing so.